1. Overview of scenario data

First, I look at how participants rated the appropriateness of each violation in each scenario, and the appropriateness of the different types of punishments in that scenario.

Data is displayed separately by treatment, i.e. whether the violation is done by a friend/non-friend, and weather the punishment is performed by a friend to a friend/non-friend. Treatments are between subjects.

Insights:
All violations are considered fairly inappropriate, with insulting a family member and stealing being the worst.
There is quite some variation in appropriateness of the different types of punishment.
There are no striking differences between treatments.

Stealing

Phone

Interrupt

Insult

Avoid

1.1. Correlation of appropriantess of violations and appropriateness of punishment.

Hypothesis: more appropriate a behavior is, the less appropriate the punishment should be (neg corr), except doing nothing (pos corr)

2. Analysis of appropriateness of violations across all domains

Now I merge all the scenarios together.

Insight:

Overall appropriateness is quite low, with again no striking difference between treatments.

2.1 ANOVA

First I run an ANOVA to get an idea of differences across treatments and domains. The ANOVA showed that both treatment and domain significantly affected ratings, with a significant interaction indicating that the effect of treatment varied across domains.

##                    Df Sum Sq Mean Sq F value   Pr(>F)    
## treatment           1     12   12.50   9.080  0.00261 ** 
## domain              4    324   80.97  58.817  < 2e-16 ***
## treatment:domain    4     47   11.71   8.507 8.16e-07 ***
## Residuals        2433   3349    1.38                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2.2 Regressions

I now run two multilevel linear regressions with participants as random intercept to zoom in on size and direction of efffects.

First, a linear regression predicting rating ~ treatment shows a small effect of treatment: violations by friends are considered slightly more appropriate.

If we include a test of interactions between treatment and domain, we can see that the effect is driven by the “exclude from online chat scenario:

“One student who is part of your friends/others group (Student A) excluded another student, who is also a part of your friends group, from a group chat on WhatsApp.”

Rating ~ Treatment

  rating
Predictors Estimates CI p
(Intercept) 1.08 1.00 – 1.16 <0.001
treatmentf 0.14 0.05 – 0.23 0.003
Random Effects
σ2 1.26
τ00 ID2 0.27
ICC 0.18
N ID2 496
Observations 2443
Marginal R2 / Conditional R2 0.003 / 0.178

Rating ~ Treatment * Domain

  rating
Predictors Estimates CI p
(Intercept) 0.61 0.46 – 0.75 <0.001
treatmentf -0.04 -0.25 – 0.17 0.721
domain interrupt 0.53 0.32 – 0.74 <0.001
domain [online] 0.68 0.50 – 0.87 <0.001
domain stealing 0.35 0.14 – 0.56 0.001
domain [texting] 0.79 0.60 – 0.97 <0.001
treatmentf × domain
interrupt
0.13 -0.19 – 0.46 0.421
treatmentf × domain
[online]
0.74 0.48 – 1.00 <0.001
treatmentf × domain
stealing
-0.01 -0.33 – 0.32 0.970
treatmentf × domain
[texting]
0.07 -0.19 – 0.33 0.602
Random Effects
σ2 1.07
τ00 ID2 0.31
ICC 0.22
N ID2 496
Observations 2443
Marginal R2 / Conditional R2 0.103 / 0.301

3. Analysis of appropriateness of punishment if friend vs. non-friend

Now I look at the appropriateness ratings of punishments if the violator is a friend vs. non friend.

3.1 Vizualization

Again, visual inspection does not reveal any striking differences bewteen treatments and domains.

Overall appropriatess across domains

Appropriateness by domain

Appropriateness of punishment by treatment and punisment type

3.1 ANOVA

First, I run an ANOVA to get an idea of differences in mean ratings across treatments and domains.

There was no significant main effect of treatment.

In contrast, domain was significant, suggesting that the ratings varied meaningfully across different domains.

The interaction between treatment and domain was also significant, implying that the effect of treatment differed depending on the domain.

##                    Df Sum Sq Mean Sq F value Pr(>F)    
## treatment           1      0    0.00   0.002 0.9611    
## domain              4    475  118.69  68.455 <2e-16 ***
## treatment:domain    4     15    3.83   2.208 0.0656 .  
## Residuals        7284  12630    1.73                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

3.2 Regressions

Now I fit 3 linear regressions to better explore directions of effects.

First, only with treatment as independent variable, then I add interactions with domain (insult as reference) and finally I also include punishment type as exploratory variable (angry_remark as reference).

The main results are [see tabs below for full results]:

Model 1: no effect

Model2 :

  • Raters considered more appropriate to punish friends who exclude others from group chats.

Model 3:

  • The domain “texting” was associated with significantly lower ratings compared to the reference domain.
  • The punishment types “do nothing” and “gossip” led to significantly lower ratings compared to the reference punishment type.

There a few significant interactions:

  • The combination of “interrupt” domain and “avoid” punishment type was associated with lower ratings compared to the reference groups.

  • The combination of “online” domain and “do nothing” punishment type was associated with lower ratings compared to the reference groups.

  • The combination of “stealing” domain and “do nothing” punishment type was associated with lower ratings compared to the reference groups.

  • The combination of “texting” domain and “do nothing” punishment type was associated with higher ratings compared to the reference groups.

  • The combination of “stealing” domain and “gossip” punishment type was also associated with higher ratings compared to the reference groups.

Rating ~ Treatment

  rating
Predictors Estimates CI p
(Intercept) 2.54 2.48 – 2.60 <0.001
treatmentf -0.01 -0.06 – 0.05 0.833
Random Effects
σ2 1.56
τ00 ID2 0.24
ICC 0.13
N ID2 496
Observations 7294
Marginal R2 / Conditional R2 0.000 / 0.131

Rating ~ Treatment * Domain

  rating
Predictors Estimates CI p
(Intercept) 2.61 2.50 – 2.72 <0.001
treatmentf 0.01 -0.15 – 0.16 0.936
domain interrupt -0.20 -0.35 – -0.04 0.012
domain [online] -0.14 -0.27 – -0.01 0.030
domain stealing 0.36 0.21 – 0.52 <0.001
domain [texting] -0.41 -0.53 – -0.28 <0.001
treatmentf × domain
interrupt
-0.01 -0.26 – 0.23 0.907
treatmentf × domain
[online]
0.16 -0.02 – 0.34 0.078
treatmentf × domain
stealing
-0.09 -0.33 – 0.16 0.491
treatmentf × domain
[texting]
-0.09 -0.27 – 0.09 0.320
Random Effects
σ2 1.49
τ00 ID2 0.24
ICC 0.14
N ID2 496
Observations 7294
Marginal R2 / Conditional R2 0.037 / 0.171

Rating ~ Treatment * Domain* punishment type

  rating
Predictors Estimates CI p
(Intercept) 2.89 2.73 – 3.05 <0.001
treatmentf -0.11 -0.34 – 0.12 0.361
domain interrupt -0.11 -0.34 – 0.12 0.355
domain [online] -0.16 -0.38 – 0.05 0.131
domain stealing 0.14 -0.09 – 0.37 0.231
domain [texting] -0.50 -0.71 – -0.28 <0.001
punishment type avoid -0.15 -0.36 – 0.07 0.175
punishment type [gossip] -0.69 -0.91 – -0.48 <0.001
treatmentf × domain
interrupt
0.17 -0.18 – 0.52 0.329
treatmentf × domain
[online]
0.27 -0.04 – 0.57 0.084
treatmentf × domain
stealing
0.09 -0.26 – 0.44 0.605
treatmentf × domain
[texting]
0.04 -0.27 – 0.34 0.816
treatmentf × punishment
type avoid
0.17 -0.13 – 0.47 0.264
treatmentf × punishment
type [gossip]
0.17 -0.13 – 0.47 0.271
domain interrupt ×
punishment type avoid
-0.41 -0.71 – -0.11 0.007
domain [online] ×
punishment type avoid
-0.08 -0.38 – 0.22 0.614
domain stealing ×
punishment type avoid
0.31 0.01 – 0.61 0.041
domain [texting] ×
punishment type avoid
-0.04 -0.34 – 0.26 0.813
domain interrupt ×
punishment type [gossip]
0.15 -0.15 – 0.45 0.325
domain [online] ×
punishment type [gossip]
0.15 -0.15 – 0.45 0.327
domain stealing ×
punishment type [gossip]
0.36 0.06 – 0.66 0.018
domain [texting] ×
punishment type [gossip]
0.30 0.00 – 0.60 0.050
(treatmentf × domain
interrupt) × punishment
type avoid
-0.22 -0.65 – 0.20 0.303
(treatmentf × domain
[online]) × punishment
type avoid
-0.21 -0.64 – 0.22 0.338
(treatmentf × domain
stealing) × punishment
type avoid
-0.22 -0.65 – 0.20 0.300
(treatmentf × domain
[texting]) × punishment
type avoid
-0.10 -0.52 – 0.33 0.661
(treatmentf × domain
interrupt) × punishment
type [gossip]
-0.34 -0.77 – 0.08 0.115
(treatmentf × domain
[online]) × punishment
type [gossip]
-0.11 -0.54 – 0.32 0.619
(treatmentf × domain
stealing) × punishment
type [gossip]
-0.31 -0.74 – 0.11 0.152
(treatmentf × domain
[texting]) × punishment
type [gossip]
-0.28 -0.71 – 0.14 0.195
Random Effects
σ2 1.44
τ00 ID2 0.24
ICC 0.15
N ID2 496
Observations 7294
Marginal R2 / Conditional R2 0.070 / 0.205